Stable Diffusion
Contents
Stable Diffusion#
Note
Install ekorpkit package first.
Set logging level to Warning, if you don’t want to see verbose logging.
If you run this notebook in Colab, set Hardware accelerator to GPU.
!pip install ekorpkit[art] exit()
Prepare Environment#
from ekorpkit import eKonf
eKonf.setLogger("WARNING")
eKonf.set_cuda(device=4)
print("version:", eKonf.__version__)
is_notebook = eKonf.is_notebook()
is_colab = eKonf.is_colab()
print("is notebook?", is_notebook)
print("is colab?", is_colab)
if is_colab:
eKonf.mount_google_drive()
eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:4)', 'A100-SXM4-40GB (id:5)']
INFO:ekorpkit.utils.notebook:shell type: ZMQInteractiveShell
INFO:ekorpkit.utils.notebook:Google Colab not detected.
version: 0.1.40.post0.dev2
is notebook? True
is colab? False
environment variables:
{'CUDA_DEVICE_ORDER': 'PCI_BUS_ID',
'CUDA_VISIBLE_DEVICES': '4, 5',
'EKORPKIT_CONFIG_DIR': '/workspace/projects/ekorpkit-book/config',
'EKORPKIT_DATA_DIR': None,
'EKORPKIT_LOG_LEVEL': 'INFO',
'EKORPKIT_PROJECT': 'ekorpkit-book',
'EKORPKIT_WORKSPACE_ROOT': '/workspace',
'KMP_DUPLICATE_LIB_OK': 'TRUE',
'NUM_WORKERS': 230}
Create a stable diffusion instance#
cfg = eKonf.compose("model/stable_diffusion")
sd = eKonf.instantiate(cfg)
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.utils.notebook:shell type: ZMQInteractiveShell
INFO:ekorpkit.base:setting environment variable CACHED_PATH_CACHE_ROOT to /workspace/.cache/cached_path
INFO:ekorpkit.base:setting environment variable KMP_DUPLICATE_LIB_OK to TRUE
INFO:absl:Unable to initialize backend 'tpu_driver': NOT_FOUND: Unable to find driver in registry given worker:
INFO:absl:Unable to initialize backend 'rocm': NOT_FOUND: Could not find registered platform with name: "rocm". Available platform names are: Host CUDA Interpreter
INFO:absl:Unable to initialize backend 'tpu': module 'jaxlib.xla_extension' has no attribute 'get_tpu_client'
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.models.art.base:> downloading models...
INFO:ekorpkit.models.art.base:> loading modules...
INFO:ekorpkit.models.art.base:No modules to load
INFO:ekorpkit.models.art.base:> loading models...
Generate images#
text_prompts = "a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape"
batch_name = "cave-jeju"
sd.imagine(
text_prompts,
batch_name=batch_name,
num_samples=2,
show_collage=True,
guidance_scale=2
)
INFO:ekorpkit.models.art.base:Merging config with args: {}
INFO:ekorpkit.models.art.base:Prompt: None
INFO:ekorpkit.models.art.base:Saving config to /workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(5)_settings.yaml
{'image_filepaths': ['/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(5)_0000.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(5)_0001.png'],
'config_file': 'cave-jeju(5)_settings.yaml',
'config': {'batch_name': 'cave-jeju',
'mode': 'generate',
'text_prompts': {0: ['a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape']},
'batch_num': 5,
'num_samples': 2,
'num_images_per_prompt': 1,
'width': 768,
'height': 512,
'num_inference_steps': 100,
'guidance_scale': 2,
'init_image': None,
'mask_image': None,
'inpaint_strength': 0.75,
'set_seed': 'random_seed',
'seed': 1737952845,
'resume_run': False,
'run_to_resume': 'latest'}}
Inpaint images#
base_url = "https://github.com/entelecheia/ekorpkit-book/raw/main/assets/figs"
init_image_path = f"{base_url}/chu-horse.png"
mask_image_path = f"{base_url}/chu-horse-masking-rough.png"
mask_image_exact_path = f"{base_url}/chu-horse-masking-exact.png"
init_image = eKonf.load_image(init_image_path)
mask_image = eKonf.load_image(mask_image_path)
mask_image_exact = eKonf.load_image(mask_image_exact_path)
sd.compare_images([mask_image, init_image, mask_image_exact], resize_ratio=0.5)
prompt = "a golden statue of an eagle with clouds, colorful painting by Tanya Hern, mural on the roof, digital art, artwork, beautiful, colorful, visual art on landscape, surrealism, watercolor, vivid by stunning"
images = sd.imagine(
prompt,
mode="inpaint",
init_image=init_image_path,
mask_image=mask_image_path,
num_samples=2,
num_images_per_prompt=2,
guidance_scale=3,
num_inference_steps=100,
return_including_init_image=False
)
INFO:ekorpkit.models.art.base:Merging config with args: {}
INFO:ekorpkit.models.art.base:Prompt: None
INFO:ekorpkit.models.art.base:Saving config to /workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(14)_settings.yaml
Stitch images#
sd.imagine(
text_prompts,
mode="stitch",
batch_name=batch_name,
num_samples=6,
show_collage=True,
)
INFO:ekorpkit.utils.notebook:shell type: ZMQInteractiveShell
INFO:ekorpkit.models.art.stable: >> elapsed time to imagine: 0:02:30.436980
INFO:ekorpkit.models.art.base:Saving config to /workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_settings.yaml
{'stitched_image_path': '/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_panorama.png',
'image_filepaths': ['/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0000.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0001.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0002.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0003.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0004.png',
'/workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(6)_0005.png'],
'config_file': 'cave-jeju(6)_settings.yaml',
'config': {'batch_name': 'cave-jeju',
'mode': 'Stitch',
'text_prompts': {0: ['a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape']},
'batch_num': 6,
'num_samples': 6,
'num_images_per_prompt': 1,
'width': 768,
'height': 512,
'num_inference_steps': 100,
'guidance_scale': None,
'init_image': None,
'mask_image': None,
'inpaint_strength': 0.75,
'set_seed': 'random_seed',
'seed': 3536724781,
'resume_run': False,
'run_to_resume': 'latest'}}
To view the panorama, upload the image to renderstuff.
Show Configuration#
sd.show_config(batch_name=batch_name, batch_num=0)
INFO:ekorpkit.models.art.base:Loading config from /workspace/projects/ekorpkit-book/stable-diffusions/outputs/stable-diffusion/cave-jeju/cave-jeju(0)_settings.yaml
INFO:ekorpkit.models.art.base:Merging config with diffuse defaults
INFO:ekorpkit.models.art.base:Merging config with args: {}
{'batch_name': 'cave-jeju',
'batch_num': 3,
'guidance_scale': None,
'height': 512,
'init_image': None,
'inpaint_strength': 0.75,
'mask_image': None,
'mode': 'Stitch',
'num_images_per_prompt': 1,
'num_inference_steps': 100,
'num_samples': 6,
'resume_run': False,
'run_to_resume': 'latest',
'seed': 2034638016,
'set_seed': 'random_seed',
'text_prompts': {0: ['a living tiger inside a gorgeous cave in a futuristic '
'Jeju island']},
'width': 768}